Built-in Middleware

Ready-to-use middleware for common agent patterns.

Overview

HPD-Agent includes production-ready middleware for:

• Error handling - Circuit breakers, error tracking
• Safety - PII redaction, guardrails
• Optimization - History reduction
• Observability - Logging, telemetry
• Multimodal - Asset storage and management

Circuit Breaker

Prevents infinite loops by detecting repeated identical function calls.

Usage

var agent = await new AgentBuilder()
    .WithMiddleware(new CircuitBreakerMiddleware
    {
        MaxConsecutiveCalls = 3 // Default: 3
    })
    .BuildAsync();

Behavior

Triggers when a tool is called with identical arguments more than MaxConsecutiveCalls times consecutively.

When triggered:

1. Sets SkipToolExecution = true
2. Emits TextDeltaEvent for user visibility
3. Emits CircuitBreakerTriggeredEvent for telemetry
4. Terminates agent gracefully

Example

// User: "Search for Paris weather"
// LLM: SearchWeb("Paris weather")
// LLM: SearchWeb("Paris weather") // Same call
// LLM: SearchWeb("Paris weather") // Third time
// 🚫 Circuit breaker triggers - prevents 4th call

Configuration

new CircuitBreakerMiddleware
{
    MaxConsecutiveCalls = 5, // Allow up to 5 identical calls
    TerminationMessageTemplate = "  Stopped '{toolName}' after {count} identical calls"
}

---

Error Tracking

Tracks consecutive failures across iterations.

Usage

var agent = await new AgentBuilder()
    .WithMiddleware(new ErrorTrackingMiddleware
    {
        MaxConsecutiveErrors = 3 // Default: 3
    })
    .BuildAsync();

Behavior

Uses OnErrorAsync to increment a counter on every error, and AfterIterationAsync to reset the counter when all tools succeed.

When threshold reached:

1. Sets IsTerminated = true
2. Sets TerminationReason
3. Emits MaxConsecutiveErrorsExceededEvent for observability
4. Emits TextDeltaEvent for user visibility
5. Agent stops gracefully

State

Stores ErrorTrackingStateData in middleware state:

public sealed record ErrorTrackingStateData
{
    public int ConsecutiveFailures { get; init; }
}

Example

// Error 1 (any source) → OnErrorAsync → ConsecutiveFailures = 1
// Error 2 → ConsecutiveFailures = 2
// Iteration succeeds → AfterIterationAsync → ConsecutiveFailures = 0 (reset)
// Error 1 → ConsecutiveFailures = 1
// Error 2 → ConsecutiveFailures = 2
// Error 3 → ConsecutiveFailures = 3 → Terminates

---

Total Error Threshold

Limits total errors across entire conversation turn.

Usage

var agent = await new AgentBuilder()
    .WithMiddleware(new TotalErrorThresholdIterationMiddleware
    {
        MaxTotalErrors = 5 // Default: 5
    })
    .BuildAsync();

Behavior

Terminates when cumulative tool failures in a turn exceed threshold, regardless of successes between them.

Difference from ErrorTrackingMiddleware:

• ErrorTracking: Resets on success (3 consecutive failures)
• TotalError: Never resets (5 total failures)

Example

// Iteration 1: Tool fails → Total = 1
// Iteration 2: Tool succeeds → Total = 1 (doesn't reset)
// Iteration 3: Tool fails → Total = 2
// ...
// Iteration N: Tool fails → Total = 5 → Terminates

---

History Reduction

Reduces conversation history to manage the context window, with caching to avoid expensive re-summarization.

Usage

var agent = await new AgentBuilder()
    .WithHistoryReduction(new HistoryReductionConfig
    {
        Enabled = true,
        Strategy = HistoryReductionStrategy.Summarizing, // or MessageCounting
        CountingUnit = HistoryCountingUnit.Exchanges,    // or Messages
        TargetCount = 20,           // Keep last N exchanges/messages
        SummarizationThreshold = 5  // Trigger when N units over TargetCount
    })
    .BuildAsync();

Behavior

Runs in BeforeMessageTurnAsync (once per user message). Increments an exchange counter in AfterMessageTurnAsync.

Decision flow:

1. If RunConfig.SkipHistoryReduction is set — skip
2. If count is within threshold — skip
3. If a valid cached reduction exists — apply cache (fast, no LLM call)
4. Otherwise — invoke ChatReducer, cache result, apply to history

Strategies

Strategy	Description
MessageCounting	Drops old messages without summarization
Summarizing	Uses an LLM call to summarize removed messages

Counting Units

Unit	Description
Exchanges	User+assistant pairs (default) — more stable threshold
Messages	Raw message count

Configuration

new HistoryReductionConfig
{
    Enabled = true,
    Strategy = HistoryReductionStrategy.Summarizing,
    CountingUnit = HistoryCountingUnit.Exchanges,
    TargetCount = 20,
    SummarizationThreshold = 5,
    Behavior = HistoryReductionBehavior.Normal // or CircuitBreaker
}

CircuitBreaker behavior: terminates the agent after reduction instead of continuing — useful when you need strict context control.

Events

HistoryReductionEvent is emitted for every outcome (Skipped, CacheHit, Performed):

public sealed record HistoryReductionEvent(
    string AgentName,
    HistoryReductionStatus Status,   // Skipped | CacheHit | Performed
    HistoryReductionStrategy Strategy,
    int? OriginalMessageCount,
    int? ReducedMessageCount,
    int? MessagesRemoved,
    string? SummaryContent,
    TimeSpan? CacheAge,
    TimeSpan Duration,
    string? Reason,
    DateTimeOffset Timestamp) : AgentEvent, IObservabilityEvent;

Example

// 30 exchanges accumulated (TargetCount=20, Threshold=5 → triggers at 25)
// Cache miss → LLM summarizes exchanges 1-10
// History becomes:
// [System] You are a helpful assistant
// [Assistant] Summary: "User asked about X, Y, Z..."
// [User] Exchange 11 ...
// ... (last 20 exchanges)
// [User] Current message
// HistoryReductionEvent { Status=Performed, MessagesRemoved=20, SummaryContent="..." }

---

PII Middleware

Detects and handles Personally Identifiable Information (PII) in messages using configurable per-type strategies.

Usage

var agent = await new AgentBuilder()
    .WithMiddleware(new PIIMiddleware
    {
        EmailStrategy = PIIStrategy.Redact,    // Default: Redact
        PhoneStrategy = PIIStrategy.Mask,      // Default: Mask
        SSNStrategy = PIIStrategy.Block,       // Default: Block
        CreditCardStrategy = PIIStrategy.Block // Default: Block
    })
    .BuildAsync();

Behavior

Runs in BeforeMessageTurnAsync, scanning user messages for built-in PII types. Optionally also scans tool results (ApplyToToolResults = true).

Built-in detectors:

• Email addresses
• Credit card numbers (Luhn-validated)
• Social Security Numbers (SSN)
• Phone numbers
• IP addresses

Strategies

Each PII type has an independently configurable PIIStrategy:

Strategy	Effect
Redact	Replaces with [TYPE_REDACTED] (e.g. [EMAIL_REDACTED])
Mask	Partially hides value (e.g. j***@example.com, ***-***-1234)
Hash	Replaces with a short SHA-256 hash (e.g. <email_hash:a1b2c3d4>)
Block	Throws PIIBlockedException — entire message is rejected
Allow	Passes through unchanged

Configuration

new PIIMiddleware
{
    // Per-type strategies
    EmailStrategy = PIIStrategy.Redact,        // Default: Redact
    CreditCardStrategy = PIIStrategy.Block,    // Default: Block (high risk)
    SSNStrategy = PIIStrategy.Block,           // Default: Block (high risk)
    PhoneStrategy = PIIStrategy.Mask,          // Default: Mask
    IPAddressStrategy = PIIStrategy.Hash,      // Default: Hash

    // Scope flags
    ApplyToInput = true,                       // Default: true
    ApplyToOutput = false,                     // Default: false
    ApplyToToolResults = false,                // Default: false
}

Custom Detectors

Add domain-specific PII detection with a regex pattern:

var middleware = new PIIMiddleware();
middleware.AddCustomDetector(
    name: "EmployeeId",
    pattern: @"\bEMP-\d{6}\b",
    strategy: PIIStrategy.Mask
);

Or use an async external service:

new PIIMiddleware
{
    ExternalDetector = async (text, ct) =>
    {
        return await myPIIService.DetectAsync(text, ct);
    }
}

Events and Exceptions

PIIDetectedEvent — emitted for each PII type found (useful for audit trails):

public record PIIDetectedEvent(
    string AgentName,
    string PIIType,
    PIIStrategy Strategy,
    int OccurrenceCount,
    DateTimeOffset Timestamp) : AgentEvent, IObservabilityEvent;

PIIBlockedException — thrown when a Block strategy triggers:

try
{
    await agent.RunAsync("My SSN is 123-45-6789");
}
catch (PIIBlockedException ex)
{
    Console.WriteLine($"Blocked PII type: {ex.PIIType}");
}

Example

// Input: "My email is john@example.com and SSN is 123-45-6789"
// EmailStrategy = Redact, SSNStrategy = Block
// → throws PIIBlockedException("SSN") before message reaches LLM

// Input: "My email is john@example.com and phone is 555-1234"
// EmailStrategy = Redact, PhoneStrategy = Mask
// → "My email is [EMAIL_REDACTED] and phone is ***-***-1234"

---

Container Middleware

Scopes middleware to specific Toolkits/skills.

Usage

var agent = await new AgentBuilder()
    .WithMiddleware(new ContainerMiddleware
    {
        ToolkitName = "WebSearch",
        Middleware = new RetryMiddleware()
    })
    .BuildAsync();

Behavior

Wraps another middleware, only executing it when:

• context.ToolkitName matches ToolkitName, OR
• context.SkillName matches SkillName

Use for Toolkit/skill-specific behavior.

Example

// Only retry web search tools
new ContainerMiddleware
{
    ToolkitName = "WebSearch",
    Middleware = new RetryMiddleware { MaxRetries = 5 }
}

// Only log database operations
new ContainerMiddleware
{
    ToolkitName = "Database",
    Middleware = new LoggingMiddleware()
}

---

Document Handling Middleware

Processes document-related operations.

Usage

var agent = await new AgentBuilder()
    .WithMiddleware(new DocumentHandlingMiddleware())
    .BuildAsync();

Behavior

Intercepts document operations and processes them according to agent configuration.

Note: Marked as legacy. For new code, use the document Skills instead — see the Skills documentation for the recommended approach.

---

Asset Upload Middleware

Automatically uploads binary assets (images, audio, PDFs) to storage and transforms DataContent → UriContent references.

Usage

Auto-registered - No configuration needed! This middleware is automatically added by AgentBuilder when you have a session store with asset support.

// AssetUploadMiddleware is automatically registered
var store = new JsonSessionStore("./data"); // Has built-in LocalFileAssetStore

var agent = await new AgentBuilder()
    .WithProvider("openai", "gpt-4o")
    .WithSessionStore(store)
    .BuildAsync();

await agent.CreateSessionAsync("session-id");

// AssetUploadMiddleware automatically:
// 1. Uploads imageBytes to store.AssetStore
// 2. Replaces DataContent with UriContent (asset://abc123)
// 3. Emits AssetUploadedEvent
var message = new ChatMessage(ChatRole.User, [
    new TextContent("What's in this image?"),
    new DataContent(imageBytes, "image/png")
]);

await foreach (var evt in agent.RunAsync(message, sessionId: "session-id"))
{
    // Handle events
}
// Session now contains UriContent reference, not raw bytes — auto-saved after turn

Behavior

Runs in BeforeIterationAsync:

1. Checks if session.Store.AssetStore exists (zero-cost exit if null)
2. Scans messages for DataContent with binary data
3. Uploads each asset to session.Store.AssetStore
4. Replaces DataContent with UriContent using asset:// URI scheme
5. Updates both context messages (for LLM) AND session messages (for persistence)
6. Emits AssetUploadedEvent or AssetUploadFailedEvent

Zero-Cost Abstraction

If session.Store.AssetStore is null, the middleware returns immediately with zero overhead:

// No AssetStore - middleware does nothing (zero cost)
var agent = await new AgentBuilder()
    .WithProvider("openai", "gpt-4o")
    .BuildAsync(); // InMemorySessionStore has no AssetStore

await agent.CreateSessionAsync("id");

// DataContent passes through unchanged
await foreach (var evt in agent.RunAsync(
    new ChatMessage(ChatRole.User, [new DataContent(imageBytes, "image/png")]),
    sessionId: "id"))
{ }

Supported Asset Types

Any binary content via DataContent:

• Images: PNG, JPEG, GIF, WebP
• Audio: MP3, WAV, OGG
• Documents: PDF, DOCX
• Videos: MP4, WebM
• Any other: application/octet-stream

Events

AssetUploadedEvent:

public record AssetUploadedEvent(
    string AssetId,
    string MediaType,
    int SizeBytes
) : AgentEvent;

AssetUploadFailedEvent:

public record AssetUploadFailedEvent(
    string MediaType,
    string Error
) : AgentEvent;

Example: Vision Model

var agent = await new AgentBuilder()
    .WithProvider("openai", "gpt-4o") // Vision model
    .WithSessionStore("./data")
    .BuildAsync();

await agent.CreateSessionAsync("vision-chat");

// Add image from file
var imageBytes = await File.ReadAllBytesAsync("photo.jpg");
var message = new ChatMessage(ChatRole.User, [
    new TextContent("What's in this photo?"),
    new DataContent(imageBytes, "image/jpeg")
]);

// Asset automatically uploaded, message transformed, session auto-saved
await foreach (var evt in agent.RunAsync(message, sessionId: "vision-chat"))
{
    if (evt is AssetUploadedEvent upload)
        Console.WriteLine($"Uploaded {upload.MediaType}: {upload.AssetId}");
}

Example: Multiple Assets

var message = new ChatMessage(ChatRole.User, [
    new TextContent("Compare these:"),
    new DataContent(image1, "image/png"),
    new TextContent("versus"),
    new DataContent(image2, "image/jpeg"),
    new TextContent("and this PDF:"),
    new DataContent(pdfBytes, "application/pdf")
]);

// All three assets uploaded automatically
await foreach (var evt in agent.RunAsync(message, sessionId: "vision-chat"))
{
    if (evt is AssetUploadedEvent upload)
        Console.WriteLine($"Asset {upload.AssetId}: {upload.SizeBytes} bytes");
}

Session Persistence

The key benefit: sessions store URI references instead of binary data.

Before transformation (in-memory):

{
  "role": "user",
  "contents": [
    { "type": "text", "text": "What's in this image?" },
    { "type": "data", "data": "iVBORw0KG...", "mediaType": "image/png" }
  ]
}

After transformation (persisted):

{
  "role": "user",
  "contents": [
    { "type": "text", "text": "What's in this image?" },
    { "type": "uri", "uri": "asset://abc123", "mediaType": "image/png" }
  ]
}

The asset is stored separately in ./data/assets/abc123.png.

Custom Asset Store

Implement IAssetStore for custom storage (S3, Azure Blob, database):

public class S3AssetStore : IAssetStore
{
    public async Task<string> UploadAssetAsync(
        byte[] data,
        string contentType,
        CancellationToken ct = default)
    {
        var assetId = Guid.NewGuid().ToString("N");
        await _s3Client.PutObjectAsync(new PutObjectRequest
        {
            BucketName = _bucketName,
            Key = assetId,
            InputStream = new MemoryStream(data),
            ContentType = contentType
        }, ct);
        return assetId;
    }

    public async Task<AssetData> DownloadAssetAsync(
        string assetId,
        CancellationToken ct = default)
    {
        var response = await _s3Client.GetObjectAsync(_bucketName, assetId, ct);
        using var ms = new MemoryStream();
        await response.ResponseStream.CopyToAsync(ms, ct);
        return new AssetData(assetId, ms.ToArray(), response.Headers.ContentType);
    }
}

// Use with session store
public class MySessionStore : ISessionStore
{
    public IAssetStore? AssetStore => new S3AssetStore();
    // ... other methods
}

---

Retry (RetryMiddleware)

Retries failed function and model calls with configurable backoff strategies.

Usage

var agent = await new AgentBuilder()
    .WithMiddleware(new RetryMiddleware(new ErrorHandlingConfig
    {
        MaxRetries = 3,
        RetryDelay = TimeSpan.FromSeconds(1),
        BackoffMultiplier = 2.0,
        MaxRetryDelay = TimeSpan.FromSeconds(30)
    }))
    .BuildAsync();

Behavior

Uses a 3-tier priority system for retry decisions:

1. Custom Strategy — CustomRetryStrategy delegate if provided
2. Provider-Aware — Respects Retry-After headers from LLM providers
3. Exponential Backoff — Falls back to RetryDelay * BackoffMultiplier^attempt

Hooks: WrapFunctionCallAsync and WrapModelCallStreamingAsync. Emits FunctionRetryEvent and ModelCallRetryEvent (both IObservabilityEvent) for each attempt.

Configuration

new ErrorHandlingConfig
{
    MaxRetries = 3,
    RetryDelay = TimeSpan.FromSeconds(1),
    BackoffMultiplier = 2.0,
    MaxRetryDelay = TimeSpan.FromSeconds(30),
    IncludeDetailedErrorsInChat = false,

    // Per-category limits (optional)
    MaxRetriesByCategory = new()
    {
        [ErrorCategory.RateLimit] = 5,
        [ErrorCategory.Network] = 2
    },

    // Fully custom strategy (optional)
    CustomRetryStrategy = async (ex, attempt, ct) =>
    {
        if (ex is RateLimitException rle)
            return rle.RetryAfter;
        return null; // null = fall through to provider/backoff
    }
}

---

Function Timeout (FunctionTimeoutMiddleware)

Enforces a maximum execution time per function call.

Usage

var agent = await new AgentBuilder()
    .WithMiddleware(new FunctionTimeoutMiddleware(TimeSpan.FromSeconds(30)))
    .BuildAsync();

Behavior

Hook: WrapFunctionCallAsync. Wraps each function call with Task.WaitAsync(timeout). If the function doesn't complete in time, throws TimeoutException with the function name and timeout duration in the message.

Recommended placement: Inside RetryMiddleware (so timeouts can be retried), outside permissions middleware (so timeout occurs before permission checks).

.WithMiddleware(new RetryMiddleware(config))          // outer — retries timeouts
.WithMiddleware(new FunctionTimeoutMiddleware(30s))   // inner — enforces per-call limit

---

Error Formatting (ErrorFormattingMiddleware)

Sanitizes exception messages before they reach the LLM, preventing leakage of stack traces, paths, connection strings, and API keys.

Usage

var agent = await new AgentBuilder()
    .WithMiddleware(new ErrorFormattingMiddleware())       // safe defaults
    // or
    .WithMiddleware(new ErrorFormattingMiddleware(new ErrorHandlingConfig
    {
        IncludeDetailedErrorsInChat = true  // include full exception messages
    }))
    .BuildAsync();

Behavior

Hooks: WrapFunctionCallAsync, AfterFunctionAsync, WrapModelCallStreamingAsync.

IncludeDetailedErrorsInChat	What LLM sees
false (default)	"Error: Function 'DeleteFile' failed."
true	Full exception message (useful for debugging, not production)

Always use false in production to prevent internal details from leaking into conversation history and LLM responses.

---

Logging (LoggingMiddleware)

Structured logging of agent execution at message turn, iteration, and function levels.

Usage

var agent = await new AgentBuilder()
    .WithMiddleware(new LoggingMiddleware(loggerFactory))
    // or with options
    .WithMiddleware(new LoggingMiddleware(loggerFactory, LoggingMiddlewareOptions.Minimal))
    .BuildAsync();

Preset Options

Preset	What It Logs
LoggingMiddlewareOptions.Default	Message turns + functions with full details
LoggingMiddlewareOptions.Minimal	Function names and timing only — no args/results
LoggingMiddlewareOptions.Verbose	Everything, unlimited string length

Custom Options

new LoggingMiddlewareOptions
{
    LogMessageTurn = true,       // Log at message turn level
    LogIteration = true,         // Log before/after each LLM call
    LogFunction = true,          // Log before/after each function
    IncludeTiming = true,        // Include execution timing
    IncludeArguments = true,     // Include function arguments
    IncludeResults = true,       // Include function results
    IncludeInstructions = true,  // Include system instructions
    MaxStringLength = 1000,      // Truncate long strings (0 = unlimited)
    LogPrefix = "[HPD-Agent]"    // Prefix for all log messages
}

Hooks: BeforeMessageTurnAsync, AfterMessageTurnAsync, BeforeIterationAsync, AfterIterationAsync, BeforeFunctionAsync, AfterFunctionAsync.

---

Combining Middleware

Stack multiple middleware for layered behavior:

var agent = await new AgentBuilder()
    .WithProvider("openai", "gpt-4o")
    .WithToolkit<MyTools>()
    // Layer 1: Redact PII
    .WithMiddleware(new PIIMiddleware())
    // Layer 2: Reduce history
    .WithMiddleware(new HistoryReductionMiddleware { MaxHistoryTokens = 3000 })
    // Layer 3: Track errors
    .WithMiddleware(new ErrorTrackingMiddleware { MaxConsecutiveErrors = 3 })
    // Layer 4: Circuit breaker
    .WithMiddleware(new CircuitBreakerMiddleware { MaxConsecutiveCalls = 3 })
    .BuildAsync();

Execution order:

• Before hooks: PIIMiddleware → HistoryReduction → ErrorTracking → CircuitBreaker
• After hooks: CircuitBreaker → ErrorTracking → HistoryReduction → PIIMiddleware

---

Custom Configuration

All middleware support property-based configuration:

var circuitBreaker = new CircuitBreakerMiddleware
{
    MaxConsecutiveCalls = 5,
    TerminationMessageTemplate = "Custom message: {toolName} called {count} times"
};

var errorTracking = new ErrorTrackingMiddleware
{
    MaxConsecutiveErrors = 2
};

var agent = await new AgentBuilder()
    .WithMiddleware(circuitBreaker)
    .WithMiddleware(errorTracking)
    .BuildAsync();

---

Built-in Middleware Summary

Middleware	Purpose	Hook	State	Auto-registered
AssetUploadMiddleware	Upload binary assets	BeforeIteration	None	Yes
ContainerMiddleware	Tool collapsing & skill scoping	Multiple	ContainerMiddlewareState	Yes (auto)
CircuitBreakerMiddleware	Prevent infinite loops	BeforeToolExecution, AfterIteration	CircuitBreakerState	No
ErrorTrackingMiddleware	Track consecutive errors	OnError, AfterIteration	ErrorTrackingStateData	No
TotalErrorThresholdIterationMiddleware	Limit total errors	AfterIteration	None	No
HistoryReductionMiddleware	Reduce message history	BeforeMessageTurn, AfterMessageTurn	HistoryReductionStateData	No
PIIMiddleware	Detect and handle PII	BeforeMessageTurn, AfterIteration	None	No
RetryMiddleware	Retry failed calls with backoff	WrapFunctionCall, WrapModelCallStreaming	None	No
FunctionTimeoutMiddleware	Enforce per-call time limit	WrapFunctionCall	None	No
ErrorFormattingMiddleware	Sanitize errors before LLM	WrapFunctionCall, AfterFunction	None	No
LoggingMiddleware	Structured execution logging	Multiple	None	No
DocumentHandlingMiddleware	Process documents (Legacy)	Various	None	No

---

Next Steps

• 05.1 Middleware Lifecycle - Understand when each hook fires
• 05.2 Middleware State - See how state works in ErrorTracking
• 05.3 Middleware Events - Learn event emission (used in CircuitBreaker)
• 05.5 Custom Middleware - Build your own middleware